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DSB-Proposed  IT  Acquisition  Model 
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“An  analysis  of  32  major  automated  information  system  acquisitions... 
the  average  time  to  deliver  an  initial  program  capability  is  ...’ 
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Proposed  IT  Acquisition  Model 


Capability 
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New  process,  modeled  on  successful  commercial  practices 

•  Rapid,  delivery  of  initial  capability  in  ~12  months  or  less* 

•  Multiple  rapidly  executed  increments  of  capability 

•  Capabilities  are  sub-divided  into  multiple  increments 

•  Tested  and  deployable  if  deemed  suitable  and 
survivable 

•  “lnteCif^i>cTOJBoard:  DoD  Policies  and  Procedures  for 

the  Acquisition  of  Information  Technology,  March  2009 

*  Current  T&E  processes  won’t 
support  IOC  in  12  months  or  less 
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DoD  Goal  for  Integrated  T&E 


*  “The  goal  of  integrated  testing  is  to  conduct  a 
seamless  test  program  that  produces  credible 
qualitative  and  quantitative  data  useful  to  all 
evaluators,  and  to  address  developmental, 
sustainment,  and  operational  issues” 

*  “Integrated  testing  allows  for  the  collaborative 
planning  of  test  events,  where  a  single  test  point  or 
mission  can  provide  data  to  satisfy  multiple 
objectives,  without  compromising  the  test  objectives 
of  participating  test  organizations.” 

From  Interim  Defense  Acquisition  Guide  (DAG),  15  June  2009 


DISTRIBUTION  STATEMENT  A  -  Cleared  for  public  release 
by  OSR  on  27  Apr  2010  -  SR  case  number  10-S-1802 


UNCLASSIFIED 


4 


Establish  Integrated  T&E 


•  The  way  ahead  to  “Establish  Integrated  T&E” 

-  Establish  an  Integrated  Test  Team  (ITT) 

-  Design  an  Integrated  T&E  Model  based  on  the 
program’s  Acquisition  Category  (ACAT)  and  System 
Development  Life  Cycle  (SDLC)  phase 

-  Initiate  innovative  approaches  to  testing 

-  Develop  an  inclusiveness  across  the  test  continuum 

-  Implement  significant  changes  to  the  current  T&E 
process 
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Establish  an  ITT 


•  Establish  the  ITT  Charter  to  define  roles  and 
responsibilities  of  participating  members: 

-Developmental  Test  &  Evaluation  (DT&E) 
-Operational  Test  &  Evaluation  (OT&E) 

-Information  Assurance  (IA) 

-Interoperability  (IOP) 

-Functional  Proponent  (FP) 

-Program  Management  Office  (PMO) 

-  System  Engineering  (SE) 

-  Configuration  Management  (CM) 
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An  Integrated  T&E  Model 
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Integrated  T&E  Model  Example 

(cont.) 


INTEGRATED  DT/OT  “ WINDOW ” 
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Desirable  features  of  an  Integrated 

T&E  Model 


•  Establish  the  ITT  -  who  is  on  it,  who  chairs  it,  when  was  it 
formed  (will  be),  what  is  its  charter,  how  often  will  the  charter  be 
updated,  and  how  often  should  the  ITT  meet? 

•  Establish  testing  strategies  based  on  risk  assessments  and 
capability  definitions 

•  Define  events  and  be  event-driven 

-  All  events  must  contain  action  lists  that  are  characterized  by  entrance  and 
exit  criteria 

•  Evaluation  methods  and  risk  management  approaches  to  govern 
the  intensity  of  test  events,  to  include  when  to  do  performance 
testing  following  large  or  small  releases 

•  Define  the  test  program’s  data  management  strategy 

-  What  data  elements  need  to  be  collected  as  well  as  when,  and  why  they 
are  needed 

-  What  tools  are  being  used  to  share  and  leverage  data  amongst  all  T&E 
activities  and  organizations 
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Establish  Risk-Based  Testing 

Strategies 


•  Existing  OSD  policy  supports  risk-based  T&E,  to  include  support 
for  T&E  of  envisioned  rapid  or  agile  IT  acquisitions 

-  Examples  are  DOT&E  Guidelines  for  OT&E  of  Software  Intensive 
Systems,  IA,  and  Net  Ready  Key  Performance  Parameter  (NR-KPP) 

•  However,  significant  reengineering,  reorganization,  and 
resourcing  at  the  DoD  Component  level  is  needed  to  fully 
implement  rapid  or  agile  IT  T&E  across  the  enterprise 

-  Scalable,  repeatable,  rigorous  T&E  processes  are  required  that  are 
aligned  to  the  assessed  risk-based  levels  of  testing  for  agile  IT  cycle  times 

-  Establishment  of  DoD  Component  Responsible  Test  Organizations 
(RTOs)  needed  to  provide  sufficient  agile  IT  T&E  expertise 

-  Investment  in  tools  required  to  support  agile  T&E  planning,  management, 
execution,  reporting,  and  oversight  efforts  across  the  enterprise 

-  Examples  include  information  portals,  automated  test  tools,  and  workflow 
management  tools,  that  are  accessible  to  the  cognizant  T&E  enterprise 
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Desirable  features  of  an  Integrated 

T&E  Model  (cont.) 


•  A  transition  plan  for  established  programs 

•  Product  processes  and  product  templates 

-  Briefing  guides 

-  Test  plan 

-  Report  guides 

-  RFP  verbiage 

-  Tool  kits 

•  Metrics,  data  sources,  and  collection  methods  that  are 
measurable,  aggressive  yet  realistic 

•  T&E  budgeting  process  with  rules  of  thumb  driven  by: 

-  Historical  data 

-  Test  category  of  Work  Breakdown  Structure  elements,  i.e. ,  custom 
development,  modified  and/or  existing  COTS  and/or  GOTS  development, 
and/or  integration  effort 

•  Establish  detailed  lists  of  documentation  required  to  support  T&E 
planning  and  execution 
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Integrated  T&E  Model  should  focus 
on  Critical  Risk  Areas 
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Summary 


*  Integrated  T&E  Model  success  is  based  on  active 
support  of  all  agencies  implementing  an  Integrated 
T&E  Model  for  IT  systems 

-  Needed  to: 

-  Be  responsive  to  rapid,  iterative  development  and  evolving 
requirements/priorities 

-  Provide  testing  that  is  appropriate  to  the  identified  level  of  program  risk 
and  capability  based 

-  Facilitate  T&E  planning,  execution,  and  reporting  timeframes  that 
accommodate  the  rapid  deployment  strategy,  e.g.,  using  automated 
test  tools 

-  Identify  and  commit  users  to  early  participation 

*  Additional  analysis  is  needed  to  determine  if  changes 
are  required  to  Title  10  and  DoDI  5000.02  to  facilitate  a 
seamless  and  more  efficient  Integrated  T&E  Model 
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Questions? 
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Contact 


OFFICE  OF  THE  SECRETARY  OF  DEFENSE 
DIRECTOR,  DEFENSE  RESEARCH  AND 
ENGINEERING 

DEVELOPMENTAL  TEST  &  EVALUATION 


3090  Defense  Pentagon 
Room  3B941 

Washington,  DC  20301-3090 

Email:  ddre-dte@osd.mil 

www.acq.osd.mil/dte 


The  right  information,  to  the  right  decision  maker,  at  the  right  time,  for 

better  decisions 
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